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IMAGE PROCESSING APPARATUS AND METHOD 

The present application claims priority to 
Japanese Patent Application No. 11-229181 filed August 13, 
1999, the entire content of which is hereby incorporated 
by reference. 

BACKGROUND OF THE INVENTION 
FIELD OF THE INVENTION 

The present invention relates to a digital 
image edge enhancement process. 

DESCRIPTION OF THE RELATED ART 

Digital images input from a digital camera and 
the like are subjected to various types of image 
correction. Various methods are used in the edge 
enhancement process, which is one of these image 
correction processes. For example, the edge is 
determined to be between pixels in the image process 
disclosed in Japanese Laid-Open Patent No. 5-108823. 
Furthermore, a threshold value which determines the edge 
is automatically calculated in the image process 
disclosed in Japanese Laid-Open Patent 10-143673. 

Generally, when enhancing the sharpness of a 
digital image by an edge enhancement process, it is 



desirable to adjust the thickness of the edge, i.e., the 
range" of edge enhancement, in accordance with the 
characteristics of the photographic object. For example, 
when photographing a building or man-made object, the 
image appears to have an added border when the line of 
the contour of the building is thickened, such that the 
image appears unnatural. Accordingly, it is desirable to 
enhance the line by narrowing the edge enhancement range 
so as to not thicken the line in the case of buildings 
and the like. Conversely, when enhancing an edge by 
thinning the edge in the case of a human image and the 
like, the enhancement often produces the opposite effect 
of making the eyes and the like appear unnatural. For 
this reason it is desirable to smoothly enhance the edge 
by broadening the width of the edge in the case of human 
images. That is, when photographing people, the 
enhancement range must be somewhat broadened with 
moderate edge enhancement. Numerous methods of automatic 
edge enhancement have been proposed heretofore, but the 
enhancement range must be input manually. 

SUMMARY OF THE INVENTION 

An object of the present invention is to 
provide an image processing apparatus capable of 
automatically accomplishing suitable edge enhancement. 



Another object of the present invention is to 
provide an image processing apparatus capable of 
excellent edge enhancement for human images as well as 
image of man-made objects. 

These and other objects are attained by an 
image processing apparatus, comprising edge detecting 
means for determining the presence/absence of an edge at 
each pixel of input image data; selecting means for 
selecting a weighting matrix corresponding to the 
position of the edge of each target pixel determined to 
have an edge by the edge detecting means; calculating 
means for calculating data of the target pixel and the 
pixels surrounding the target pixel using the weighting 
matrix selected by the selecting means; enhancement range 
determining means for determining the range of edge 
enhancement of the target pixel by comparing the 
calculation result of the calculating means to a specific 
threshold value; and edge enhancing means for executing 
an edge enhancement process on data of object pixels 
within the enhancement range determined by the 
enhancement range determining means . 

The present invention is a medium readable by a 
computer storing computer-executable programs comprising 
the steps of determining the presence/absence of an edge 
at each pixel of input image data; selecting a weighting 



matrix corresponding to the position of the edge for each 
target pixel determined to have an edge; calculating data 
of the target pixel and pixels surrounding the target 
pixel using selected weighting matrix; comparing the 
calculation result to a specific threshold value; 
determining the range of edge enhancement for the target 
pixel based on the comparison result; and executing an 
edge enhancement process for the object pixels within the 
determined enhancement range. 

It is desirable that the enhancement range 
determining means increases the weighting of components 
corresponding to the interior side of the edge in the 
weighting matrix. 

It is further desirable that the enhancement 
range determining means selects a weighting matrix based 
on the presence/absence of an edge in four directions 
surrounding the target pixel. 

It is further desirable that the edge enhancing 
means executes processing based on the hue and chroma of 
the pixels surrounding the object pixel. 

It is even further desirable that the edge 
enhancing means executes processing based on the distance 
of the object pixel to the target pixel. 

The invention itself, together with further 
objects and attendant advantages, will be best understood 



by reference to the following detailed description taken 
in conjunction with the accompanying drawings. 



BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 shows the overall construction of an 
image processing apparatus; 

FIG. 2 is a block diagram of the image 
processing apparatus; 

FIG. 3 is a flow chart of edge enhancement; 

FIG. 4 shows the coordinates within a block of 
3x3 pixels; 

FIGS. 5A through 5D show the edge detection 

matrices ; 

FIG. 6 shows one example of an enhanced pixel; 
FIG. 7 shows the target pixel and the edge 

position; 

FIG. 8 shows the target pixel and the edge 

position; 

FIG. 9 shows the target pixel and the edge 

position; 

FIG. 10 shows the target pixel and the edge 

position; 

FIG. 11 shows the coordinates within a block of 
5x5 pixels; 



FIGS. 12A through 12D show examples of 
weighting matrices ; and 

FIG. 13 shows an edge enhancement matrix. 

In the following description , like parts are 
designated by like reference numbers throughout the 
several drawings. 

DETAILED DESCRIPTRION OF THE PREFERRED EMBODIMENTS 

The embodiments of the present invention are 
described hereinafter with reference to the accompanying 
drawings . 

The image processing apparatus of the present 
embodiment of the invention has a controller 1 at it's 
center for controlling the overall image processing 
apparatus provided with a CPU. In FIG. 1, the arrows 
represent the flow of data. A display 2 displays images 
or text and the like, and displays various types of 
screens for operation. A keyboard 3 and mouse 4 are used 
for various input and specified operations. A floppy 
disk 5 and hard drive 6 are data storage media for 
storing and saving image data, and the image processing 
apparatus is provided with a floppy disk drive and a hard 
disk drive for accessing the floppy disk 5 and the hard 
disk 6. A printer 7 prints the image data and the like 
on paper. A scanner 8 reads image data from a document. 



A CD-ROM 9 is a storage medium for storing large amounts 
of various types of image data, and the image processing 
apparatus is provided with a CD-ROM drive to access the 
CD-ROM 9. A speaker 10 for audio output and a microphone 
11 for audio input are connected. An image processing 
program described later is read from an external storage 
medium such as the CD-ROM. 

FIG. 2 is a block diagram centered on a 
controller 1. The controller 1 has a CPU 2 01 at it 
center, and connected to this CPU 201 via a data bus 22 0 
are a ROM 203 for storing various processing programs and 
the like, RAM 204 for storing programs and various data, 
display control circuit 205 for displaying image and text 
and the like on the display 2, keyboard control circuit 
206 for controlling and transferring input from the 
keyboard, mouse control circuit 2 07 for controlling and 
transferring input from the mouse, floppy disk drive 
control circuit 208 for controlling the floppy disk drive, 
hard disk control circuit 209 for controlling the hard 
disk, printer control circuit 210 for controlling output 
to the printer 7, scanner control circuit 211 for 
controlling the scanner 8, CD-ROM drive control circuit 
212 for controlling the CD-ROM drive, speaker control 
circuit 213 for controlling the speaker 10, and a 
microphone control circuit 214 for controlling the 



microphone 11. Furthermore, a clock circuit 202 supplies 
various types of clock signals necessary for the 
operation of the image processing apparatus. An 
expansion slot 215 is connected so as to allow connection 
of various types of expansion ports via the data bus. 

The controller 1 performs various types of 
image processing on a digital image. The digital image 
edge enhancement process is described below. FIG. 3 
shows the flow of the edge enhancement process performed 
by the controller 1 . 

First, image data in the RGB color system are 
converted to image data in the HSL color system (step 
S10). Then, steps S11-S18 are steps of the edge 
enhancement process performed on the L component of the 
image data in the HSL color system of a specific target 
pixel, and these steps are repeated until the edge 
enhancement process has been performed on all pixels. 

In step Sll, a filtering process is executed to 
detect the edge. Specifically, a 3x3 pixel block 
comprising a target pixel and the surrounding pixels is 
extracted, and the block is filtered. The coordinates 
within this block are defined as shown in FIG. 4. The 
pixel at coordinates (2,2) is the target pixel. The edge 
detecting matrices wl used in this filtering is shown in 
FIGS. 5A through 5D. For example, FIG. 5A is a filter 



for detecting an edge present above the target pixel, and 
the edge is detected by comparing the value of Ola 
determined by equation ( 1 ) below to a specific threshold 
value . 



Ola 



(1) 



Where Xij represents the value of L at 
coordinates (i,j), and wlij represents line i column j 
component of the edge detecting matrix wl . 

Similarly, FIG. 5B is an edge detecting matrix 
for detecting an edge present below a target pixel, and 
the edge is detected by comparing the value of 81b 
determined by equation (2) below to a specific threshold 
value. 

01b- I yXwlA (2) 



Similarly, FIG. 5C is an edge detecting matrix 
for detecting an edge present to the left of a target 
pixel, and the edge is detected by comparing the value of 
01c determined by equation (3) below to a specific 
threshold value. 



die = 



(3) 



Similarly, FIG. 5D is an edge detecting matrix 
for detecting an edge present to the right of a target 
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pixel, and the edge is detected by comparing the value of 
Old determined by equation (4) below to a specific 
threshold value. 



81d = 



(4) 



In the present embodiment, the edge between 
pixels is determined as described above. In this way 
edges are naturally enhanced without enhancing only lines 
by increasing the difference between both pixels 
circumscribing the edge during enhancement. Furthermore, 
edges present around a target pixel can be limited to 
four types by determining the edge between pixels , and 
subsequent selection of weighting matrix is simply 
accomplished. 

When an edge is detected at a target pixel 
(step S12: YES), the weighting matrix w2 is selected to 
determine the enhancement radius R (step S13). Then, a 
block of 5x5 pixels comprising the target pixel and 24 
surrounding pixels is extracted, the selected weighting 
matrix w2 is used for filtering this block, and the 
enhancement radius R is determined (step S14). The 
coordinates within this block are defined as shown in FIG. 
11. FIGS. 12A through 12D show examples of the weighting 
matrices w2 . FIG. 12A shows a weighting matrix w2 when 
an edge is detected at the left side of the target pixel 
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(FIG. 7), FIG. 12B shows a weighting matrix w2 when edges 
are detected below and on the left side of the target 
pixel (FIG. 8), FIG. 12C shows a weighting matrix w2 when 
edges are detected below, on the left side and on the 
right side of the target pixel (FIG. 9), and FIG. 12D 
shows a weighting matrix w2 when an edge is detected on 
the left and right sides of the target pixel (FIG. 10). 
In all weighting matrices w2 , pixels from all to a55 are 
not 0. 

The edge enhancement radius R is a parameter 
representing the range for enhancing the edge, and is 
used to determine to which pixel to perform enhancement 
at the peripheral pixels considered as the edge. For 
example, the enhancement radius is 1 (pixel) in the 
example of FIG. 6, and the target pixel and surrounding 
pixels (shaded area) represent the enhanced pixels 
obtained by edge enhancement. 

Specifically, the enhancement radius R is 
automatically calculated in accordance with equation ( 5 ) 
below from the image data and weighting matrix w2 using 
the radius calculation function F(x). 

R=F(2x ij w2 ij ) (5) 

Where x i;j represents the value of L at 
coordinates (i,j), and w2 i:i represents the line i column j 
component of the weighting matrix w2 . 



The radius calculation function F(x) is a 
function such as shown in the examples below. 
F(x)=l when bl<x<b2 
F(x)=2 when b2<x<b3 
Where bl and b2 are constants . 

The edge enhancement range ( enhancement radius ) 
should be determined from the slope (gradation) of the 
luminance of the edge periphery and edge line type 
(thickness). The edge enhancement radius is closely 
related to the edge width. When the edge enhancement 
radius is small, the edge width is thin, and when the 
edge enhancement radius is large, the edge width is thick. 
The edge thickness must be adjusted depending on the 
object. The weighting in the weighting matrix w2 is at 
the opposite side (edge inner side) of the edge position 
relative to the target pixel. Since the edge enhancement 
radius is determined using the weighting matrix w2 , the 
determined edge enhancement radius is related to the edge 
width if an edge is in the periphery of the target pixel. 
When the object is a building, there is a slope point 
(edge) of acute change in luminance near the object edge, 
and when the object is a human image, and particularly 
the photograph of a face, the change in luminance is 
smooth near the edge. These characteristics are 
reflected in the edge enhancement radius . 
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Then, pixels within the enhancement radius R, 
i.e.,' the object pixels for edge enhancement processing, 
are subjected to the enhancement process using the 
enhancement parameters. Moreover, characteristics of the 
5 object can be reflected since the enhancement parameters 

are determined from the color information and chroma 
information near the edge. First, the enhancement 
parameter is calculated for the value L(i,j) of the image 
data of one object pixel (step S15), and the enhancement 
10 process is executed (step S16). This enhancement process 

is repeated until processing is completed for all object 
pixels within the enhancement radius R (step S17: YES). 

A first enhancement coefficient Pi and a second 
enhancement coefficient P2 are included in the 
15 enhancement parameters, and are calculated from the color 

information and the enhancement radius as described below. 

The first enhancement coefficient Pi is 
determined from the color information and chroma of the 
surroundings, i.e., from image data of the target pixel 
2 0 and the eight surrounding pixels. As shown in Table 1, 

four types of color (neutral color, flesh color, warm 
color, other) are discriminated to determine the first 
enhancement coefficient Pi . 
Table 1 
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Neutral color 


0.6 


Enhance text edge 


Flesh color 


0.7 


Produce skin texture 


Warm color 


0.7 


Soft tones are unnatural with 
hard edges 


Other 


1.0 





The second enhancement coefficient P2 is 
determined by the distance from the center pixel of the 
enhancement. As shown in Table 2, the second enhancement 
5 coefficient P2 is calculated for distances 0, 1, 2 

(pixels ) . 

Table 2 



Distance 


P2 


0 


1.0 


1 


0.8 


2 


0.6 



The corrected image data L'(i,j) are calculated 
10 by equation (6) below using the uncorrected image data 

L(i,j), the two enhancement coefficients Pi and P2, and 
the edge enhancement matrix w3 shown in FIG. 13. 

L' (i, j)=L(i, j)+W*Pl*P2 (6) 
W=2L( i, j ) *W3ij (7) 
15 Where w3ij represents the line i column j 

component of the edge enhancement matrix w3 . 



14 



Although the present invention has been fully 
described by way of examples with reference to the 
accompanying drawings , it is to be noted that various 
changes and modification will be apparent to those 
5 skilled in the art. Therefore, unless otherwise such 

changes and modifications depart from the scope of the 
present invention, they should be construed as being 
included therein. 
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WHAT IS CLAIMED IS: 

1. An image processing apparatus, comprising: 
edge detecting means for determining the 

presence/absence of an edge at each pixel of input image 
data; 

selecting means for selecting a weighting 
matrix corresponding to the position of the edge of each 
target pixel determined to have an edge by said edge 
detecting means ; 

calculating means for calculating data of the 
target pixel and the pixels surrounding the target pixel 
using the weighting matrix selected by said selecting 
means ; 

enhancement range determining means for 
determining the range of edge enhancement of the target 
pixel by comparing the calculation result of said 
calculating means to a specific threshold value; and 

edge enhancing means for executing an edge 
enhancement process on data of object pixels within the 
enhancement range determined by said enhancement range 
determining means . 

2. An image processing apparatus according to 
claim 1, wherein 
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said enhancement range determining means 
increases the weighting of components corresponding to 
the interior side of the edge in the weighting matrix. 

3. An image processing apparatus according to 
5 claim 1, wherein 

said edge detecting means determines the edge 
to be between pixels . 

4. An image processing apparatus according to 
claim 1 , wherein 

10 said enhancement range determining means 

selects the weighting matrix based on the 
presence/absence of an edge in four directions 
surrounding the target pixel. 

5. An image processing apparatus according to 
15 claim 1, wherein 

said edge enhancing means executes processing 
based on the hue and chroma of the pixels surrounding the 
object pixel. 

6. An image processing apparatus according to 
20 claim 1, wherein 

said edge enhancing means executes processing 
based on the distance of the object pixel to the target 
pixel . 

7. An image processing method, comprising the 

25 steps of: 
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determining the presence/absence of an edge at 
each pixel of input image data; 

selecting a weighting matrix corresponding to 
the position of the edge for each target pixel determined 
5 to have an edge; 

calculating data of the target pixel and pixels 
surrounding the target pixel using selected weighting 
matrix; 

comparing the calculation result to a specific 
10 threshold value; 

determining the range of edge enhancement for 
the target pixel based on the comparison result; and 

executing an edge enhancement process for the 
object pixels within the determined enhancement range. 
15 8 - An image processing method according to 

claim 7, wherein 

the weighting of components corresponding to 
the interior side of the edge in the weighting matrix is 
increased in the step of determining the range. 
2 0 9 . A medium readable by a computer storing 

computer-executable programs comprising the steps of: 

determining the presence/absence of an edge at 
each pixel of input image data; 
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selecting a weighting matrix corresponding to 
the position of the edge for each target pixel determined 
to have an edge; 

calculating data of the target pixel and pixels 
surrounding the target pixel using selected weighting 
matrix; 

comparing the calculation result to a specific 
threshold value; 

determining the range of edge enhancement for 
the target pixel based on the comparison result; and 

executing an edge enhancement process for the 
object pixels within the determined enhancement range. 

10. An image processing apparatus, comprising: 

edge detecting means for determining the 
presence/absence of an edge at each pixel of input image 
data; 

calculating means for calculating data of the 
target pixel determined to have an edge by said edge 
detecting means and the pixels surrounding the target 
pixel using the weighting matrix; 

enhancement range determining means for 
determining the range of edge enhancement of the target 
pixel by comparing the calculation result of said 
calculating means to a specific threshold value; and 



edge enhancing means for executing an edge 
enhancement process on data of object pixels within the 
enhancement range determined by said enhancement range 
determining means . 
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ABSTRACT OF THE DISCLOSURE 

Generally, when enhancing the sharpness of a 
digital image by an edge enhancement process, it is 
desirable to adjust the range of edge enhancement. For 
5 example, when photographing a building or man-made object, 

the image appears to have an added border when the line 
of the contour of the building is thickened, such that 
the image appears unnatural. Accordingly, it is 
desirable to enhance the line by narrowing the edge 

10 enhancement range so as to not thicken the line in the 

case of buildings and the like. Conversely, when 
enhancing an edge by thinning the edge in the case of a 
human image and the like, the enhancement often produces 
the opposite effect of making the eyes and the like 

15 appear unnatural. For this reason it is desirable to 

smoothly enhance the edge by broadening the width of the 
edge in the case of human images . In the image 
processing apparatus of the present invention, the edge 
enhancement range is determined by calculation using a 

20 weighting matrix selected in accordance with the 

presence/absence of an edge on whichever of the sides of 
a target pixel. Specifically, data of a target pixel and 
the surrounding pixels are calculated using a weighting 
matrix selected by a weighting selection means, and the 

25 edge enhancement range is determined by comparing this 
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calculated value to a specific threshold value. The data 
of pixels within the enhancement range determined in this 
way are subjected to the edge enhancement process. 
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K. Bingham, Reg. No. 42,327; John G. Bisbikis, Reg. No. 37,095; Daniel Bucca, Reg. No. 42,368; 
Kenneth L. Cage, Reg. No. 26,151; Stephen C. Carlson, Reg. No. 39,929; Tom A. Corrado, Reg. No. 
42,439; Paul Devinsky, Reg. No. 28,553; Laura A. Donnelly, Reg. No. 38,435; Margaret M. Duncan, Reg. 
No. 30,879; Brian E. Ferguson, Reg. No. 36,801; Michael F. Fogarty, Reg. No. 36,139; Willem F. 
Gadiano, Reg. No. 37,136; Keith E. George, Reg. No. 34,111; John A. Hankins, Reg. No. 32,029; Brian 
D. Hickman, Reg. No. 35,894; Eric J. Kraus, Reg. No. 36,190; Patrick B. Law, Reg. No. 41,549; Robert E. 
LeBlanc, Reg. No. 17,219; Jack Q. Lever, Reg. No. 28 ,149; Raphael V. Lupo, Reg. No. 28,363; 
Christine F. Martin, Reg. No. 39.762; Michael A. Messina, Reg. No. 33,424; Eugene J. Molinelli, Reg. 
No. 42,901; Christopher J. Palermo, Reg. No. 42,056; Joseph H. Paquin, Jr., Reg. No. 31,647; Robert L. 
Price, Reg. No. 22,685; Gene Z. Rubinson, Reg. No. 33,351; Joy Ann G. Serauskas, Reg. No. 27,952; 
David A. Spenard, Reg. No. 37,449; Arthur J. Steiner, Reg. No. 26,106; David L. Stewart, Reg. No. 
37,578; Michael D. Switzer, Reg. No. 39,552; Leonid D. Thenor, Reg. No. 39,397; Keith J. Townsend, 
Reg. No. 40,358; Aaron Weisstuch, Reg. No. P41,557; Edward J. Wise, Reg. No. 34,523; Alexander V. 
Yampolsky, Reg. No. 36,324; and Robert W. Zelnick, Reg. No. 36,976 all of 

Please address all correspondence and telephone calls to: 

MCDERMOTT, WILL & EMERY 
600 13th Street, N.W. 
WASHINGTON, D.C. 20005-3096 
Telephone: 202-756-8000 

I hereby declare that all statements made herein of my own knowledge are true and that all statements 
made on information and belief are believed to be true; and further that these statements were made 
with the knowledge that willful false statements and the like so made are punishable by fine or 
imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such willful 
false statements may jeopardize the validity of the application or any patent issued thereon. 

Full name of sole or first inventor Hiroki YOSHIDA 

Inventor's signature )disu>&,\ %^-?jLola~. Date .Jb£&- ^ r ^>oo 

Residence Kyoto-Shi, Kyoto, Japan 

Post Office Address c/o MINOLTA CO., LTD., Osaka Kokusai Bldg.. 3-13. 2-Chome. 

Ayuchi-Machi, Chuo-Ku. Osaka-Shi. Osaka. 541-8556 Japan Citizenship Japan 



